<html>
<head><meta charset="utf-8"><title>Extremely large statics · t-compiler · Zulip Chat Archive</title></head>
<h2>Stream: <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/index.html">t-compiler</a></h2>
<h3>Topic: <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/Extremely.20large.20statics.html">Extremely large statics</a></h3>

<hr>

<base href="https://rust-lang.zulipchat.com">

<head><link href="https://rust-lang.github.io/zulip_archive/style.css" rel="stylesheet"></head>

<a name="170671686"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/Extremely%20large%20statics/near/170671686" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> HeroicKatora <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/Extremely.20large.20statics.html#170671686">(Jul 11 2019 at 20:54)</a>:</h4>
<p>I'm trying to define a gigabyte large static via <code>MaybeUninit::uninit()</code> which leads to extreme memory usage and long compilation times in <code>rustc</code> (1GB array ~ 20GB memory) Similarly when declaring such a large static array. Somewhat understadable. But is there a way and would it be interesting to not make it do that?<br>
Note that the resulting binary is in fact very small, at least for elf, and whatever <code>ld</code> gets poses no problem to it (<code>ld</code> finishes extremely fast). The result binary merely contains a symbol for which the <em>program loader</em> needs to allocate memory, it does not contain the gigabytes of zeroed/uninitialized data. I wanted to utilize this trick for tricking a bootloader into allocating static memory for a unikernel without requiring memory management in that unikernel itself (yeah, kind of dirty hack, back to doing the same with ASM/linker script).<br>
Testable code: <a href="https://github.com/HeroicKatora/static-alloc/blob/master/tests/huuuuuge.rs" target="_blank" title="https://github.com/HeroicKatora/static-alloc/blob/master/tests/huuuuuge.rs">https://github.com/HeroicKatora/static-alloc/blob/master/tests/huuuuuge.rs</a><br>
<span class="user-mention" data-user-id="120791">@RalfJ</span> I was told to maybe ping you.</p>



<a name="170699954"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/Extremely%20large%20statics/near/170699954" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> eddyb <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/Extremely.20large.20statics.html#170699954">(Jul 12 2019 at 07:04)</a>:</h4>
<p>also pinging <span class="user-mention" data-user-id="124288">@oli</span></p>



<a name="170699977"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/Extremely%20large%20statics/near/170699977" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> eddyb <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/Extremely.20large.20statics.html#170699977">(Jul 12 2019 at 07:05)</a>:</h4>
<p>I'm pretty sure this has been brought up before, we can probably make accesses slightly more expensive by lazily allocating the data</p>



<a name="170700368"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/Extremely%20large%20statics/near/170700368" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> eddyb <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/Extremely.20large.20statics.html#170700368">(Jul 12 2019 at 07:13)</a>:</h4>
<p>the undef bitset is 1/8 the size, but I guess that could also be lazy in the same way</p>



<a name="170715266"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/Extremely%20large%20statics/near/170715266" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> oli <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/Extremely.20large.20statics.html#170715266">(Jul 12 2019 at 11:49)</a>:</h4>
<p>All zeros or all undef is easy, if everyone is OK with adding optimizations for these special cases I can write mentoring instructions</p>



<a name="170715273"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/Extremely%20large%20statics/near/170715273" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> oli <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/Extremely.20large.20statics.html#170715273">(Jul 12 2019 at 11:49)</a>:</h4>
<p>I just always gave up when thinking about sparsely initialized statics</p>



<a name="170719318"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/Extremely%20large%20statics/near/170719318" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> HeroicKatora <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/Extremely.20large.20statics.html#170719318">(Jul 12 2019 at 12:59)</a>:</h4>
<p>All undef would be sufficient. And it's more consistently justifiable instead of all zeroes where it might be more surprising why to allow exactly that case and not other repeating constant arrays like <code>[1; 1 &lt;&lt; 30]</code>. Also, would that entail being able to <code>[(0, 0); 1 &lt;&lt; 30]</code>? Maybe better to stick to undef for the moment. Maybe you can give a different view with insights from the compiler implementatino?<br>
If the mentoring instructions make it seem reasonably small and contained, I could maybe try to implement it also.</p>



<a name="170719964"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/Extremely%20large%20statics/near/170719964" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> oli <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/Extremely.20large.20statics.html#170719964">(Jul 12 2019 at 13:07)</a>:</h4>
<p>I believe that it would suffice to move the bytes and the undefmask fields of Allocation into a custom Struct and add a field of that new type to Allocation. Then once everything compiles again, make a separate commit that makes that field an Option. This will require some changes, most notably in librustc_mir/interpret. Any writes should init the option any reads should emit an undef read error if None</p>



<a name="170721926"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/Extremely%20large%20statics/near/170721926" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> HeroicKatora <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/Extremely.20large.20statics.html#170721926">(Jul 12 2019 at 13:31)</a>:</h4>
<p>That sound pretty clear cut, I'll see what I can do <span aria-label="+1" class="emoji emoji-1f44d" role="img" title="+1">:+1:</span></p>



<a name="170724425"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/Extremely%20large%20statics/near/170724425" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> nikomatsakis <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/Extremely.20large.20statics.html#170724425">(Jul 12 2019 at 14:04)</a>:</h4>
<p>(seems like all undef and all zeroes are going to be pretty common to me, so I would be <span aria-label="+1" class="emoji emoji-1f44d" role="img" title="+1">:+1:</span> to special casing them for now)</p>



<a name="170752062"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/Extremely%20large%20statics/near/170752062" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> HeroicKatora <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/Extremely.20large.20statics.html#170752062">(Jul 12 2019 at 20:11)</a>:</h4>
<p>Problematically, there seem to be <code>get</code> calls that expect being able to access the bytes despite undef. In particlar, <code>copy_repeatedly</code> in memory seems to be ok with that and even has an <code>unsafe</code> block depending on getting a byte-slice of correct length back. And <code>read_c_str</code> also tries to access the bytes directly <em>before</em> ensuring they are defined. Not sure what would break if it were to check beforehand.</p>



<a name="170753351"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/Extremely%20large%20statics/near/170753351" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> HeroicKatora <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/Extremely.20large.20statics.html#170753351">(Jul 12 2019 at 20:30)</a>:</h4>
<p>My idea would be to introduce an enum <code>BytesOrUndef&lt;'a&gt; { Bytes(&amp;'a[u8]), Undef }</code> to differentiate between these two situations where that difference is of interest. Not sure what to do with all direct accesses though. Should I post this as PR for tracking and for more comments etc?</p>



<a name="170776116"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/Extremely%20large%20statics/near/170776116" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> oli <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/Extremely.20large.20statics.html#170776116">(Jul 13 2019 at 06:04)</a>:</h4>
<p>Oh right. read_c_str can just be changed, but copy repeatedly should just not do anything if there are no defined bytes, so get can return None imo</p>



<a name="170776133"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/Extremely%20large%20statics/near/170776133" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> oli <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/Extremely.20large.20statics.html#170776133">(Jul 13 2019 at 06:05)</a>:</h4>
<p>Yea opening a PR is a good idea</p>



<a name="170776233"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/Extremely%20large%20statics/near/170776233" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> oli <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/Extremely.20large.20statics.html#170776233">(Jul 13 2019 at 06:08)</a>:</h4>
<p>Or, if you wanna prep for zeroed being special, instead of using an option for get's return type and the field type, use a generic enum like you described, but make the arbitrary data field have a generic value so you can reuse the enum for the field and return type and possibly elsewhere</p>



<a name="170779624"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/Extremely%20large%20statics/near/170779624" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> RalfJ <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/Extremely.20large.20statics.html#170779624">(Jul 13 2019 at 07:45)</a>:</h4>
<p><span class="user-mention" data-user-id="229913">@Andreas Molzer</span> <span class="user-mention" data-user-id="124288">@oli</span> with the current infrastructure however, even a 1GB static shouldnt take 20GB of memory... data + undef mask should be more like &lt;2GB, right? Doesn't this indicate we are keeping several needless copies of this data somewhere?</p>



<a name="170779889"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/Extremely%20large%20statics/near/170779889" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> oli <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/Extremely.20large.20statics.html#170779889">(Jul 13 2019 at 07:55)</a>:</h4>
<p>Well... there may be multiple locals each reserving the needed memory</p>



<a name="170779891"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/Extremely%20large%20statics/near/170779891" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> oli <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/Extremely.20large.20statics.html#170779891">(Jul 13 2019 at 07:56)</a>:</h4>
<p>But 20gb is excessive</p>



<a name="170788755"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/Extremely%20large%20statics/near/170788755" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> HeroicKatora <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/Extremely.20large.20statics.html#170788755">(Jul 13 2019 at 12:55)</a>:</h4>
<p>In that specific initial example it's not a simple static but an uninit static that's being created as part of a wrapping struct with another member. It is running through const-eval shortly.</p>



<a name="170788950"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/Extremely%20large%20statics/near/170788950" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> HeroicKatora <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/Extremely.20large.20statics.html#170788950">(Jul 13 2019 at 13:01)</a>:</h4>
<p>It could also be that the usage is from llvm. Is there an existing way that creates an undef version of a <em>complex type</em> in the llvm interface, I only found a special code path for for integer types. Instead the allocated zeroed-bytes are copied from the Allocator directly as a <code>const_bytes</code>?</p>



<a name="170789253"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/Extremely%20large%20statics/near/170789253" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> eddyb <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/Extremely.20large.20statics.html#170789253">(Jul 13 2019 at 13:09)</a>:</h4>
<p><span class="user-mention" data-user-id="124288">@oli</span> why <code>Option</code> and not just growing the backing <code>Vec</code>s as needed?</p>



<a name="170789385"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/Extremely%20large%20statics/near/170789385" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> eddyb <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/Extremely.20large.20statics.html#170789385">(Jul 13 2019 at 13:12)</a>:</h4>
<p>or does too much rely on them being of an exact size?</p>



<a name="170789407"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/Extremely%20large%20statics/near/170789407" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> HeroicKatora <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/Extremely.20large.20statics.html#170789407">(Jul 13 2019 at 13:13)</a>:</h4>
<p><span class="user-mention" data-user-id="119009">@eddyb</span> If there's supposed to be another state for constants, i.e. zeroed, then some wrapper enum is necessary. I tried out both, I had started with growing as needed before the suggestion. It seems several instances index into the vector directly and may get very confused if it is not full length. However, that could be changed as well, growing might also be doable.</p>



<a name="170789466"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/Extremely%20large%20statics/near/170789466" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> eddyb <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/Extremely.20large.20statics.html#170789466">(Jul 13 2019 at 13:15)</a>:</h4>
<p>okay</p>



<a name="170789479"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/Extremely%20large%20statics/near/170789479" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> HeroicKatora <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/Extremely.20large.20statics.html#170789479">(Jul 13 2019 at 13:15)</a>:</h4>
<p>I had to add another <code>size: Size</code> attribute either way to properly track the length of lazy allocation, so <code>.len</code> queries shouldn't be the problem</p>



<a name="170789548"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/Extremely%20large%20statics/near/170789548" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> eddyb <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/Extremely.20large.20statics.html#170789548">(Jul 13 2019 at 13:17)</a>:</h4>
<p>one kinda cool thing is that you could use this to have uninitialized &gt; 4GB arrays on 32-bit rustc</p>



<a name="170789551"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/Extremely%20large%20statics/near/170789551" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> eddyb <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/Extremely.20large.20statics.html#170789551">(Jul 13 2019 at 13:17)</a>:</h4>
<p>(by using <code>Size</code> instead of <code>usize</code>)</p>



<a name="170789607"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/Extremely%20large%20statics/near/170789607" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> HeroicKatora <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/Extremely.20large.20statics.html#170789607">(Jul 13 2019 at 13:18)</a>:</h4>
<p>Didn't really think of this, but maybe? I'm not sure because even an access into <code>uninit</code> goes through a range check that only handles up to the host architecture length. But that might be rewritten as well.</p>



<a name="170789621"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/Extremely%20large%20statics/near/170789621" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> HeroicKatora <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/Extremely.20large.20statics.html#170789621">(Jul 13 2019 at 13:19)</a>:</h4>
<p>It's up here: <a href="https://github.com/rust-lang/rust/pull/62655" target="_blank" title="https://github.com/rust-lang/rust/pull/62655">#62655</a> (Do links work like that here?)</p>



<a name="170789624"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/Extremely%20large%20statics/near/170789624" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> eddyb <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/Extremely.20large.20statics.html#170789624">(Jul 13 2019 at 13:19)</a>:</h4>
<p>yeah</p>



<a name="171273334"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/Extremely%20large%20statics/near/171273334" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> HeroicKatora <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/Extremely.20large.20statics.html#171273334">(Jul 19 2019 at 16:49)</a>:</h4>
<p>Patch seems to work: Down to 4gb allocated and 1.9gb residual now. The good question is how to best put this into a compile test.</p>



<a name="171302958"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/Extremely%20large%20statics/near/171302958" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> HeroicKatora <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/Extremely.20large.20statics.html#171302958">(Jul 20 2019 at 00:22)</a>:</h4>
<p>However, the memory usage of the raw <code>static A: MaybeUninit&lt;[u8; 1 &lt;&lt; EXP]&gt; = MaybeUninit::uninit();</code> is nowhere close to zero. I'm not sure why.</p>



<a name="171305661"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/Extremely%20large%20statics/near/171305661" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Tom Phinney <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/Extremely.20large.20statics.html#171305661">(Jul 20 2019 at 01:47)</a>:</h4>
<p>The memory usage of an array of <code>1 &lt;&lt; EXP</code> bytes should be 2&lt;sup&gt;EXP&lt;/sup&gt; bytes, whether they've been initialized or not.</p>



<a name="171306022"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/Extremely%20large%20statics/near/171306022" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> HeroicKatora <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/Extremely.20large.20statics.html#171306022">(Jul 20 2019 at 02:00)</a>:</h4>
<p>Its about memory usage of such a static <em>in the compiler</em> (that is, within mir) not in the running program. Uninitialized memory has the choice of choosing any value when it is accessed, and hence it is not necessary allocate bytes for keeping track of a current value.</p>



<a name="171314097"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/Extremely%20large%20statics/near/171314097" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> oli <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/Extremely.20large.20statics.html#171314097">(Jul 20 2019 at 06:15)</a>:</h4>
<p><span class="user-mention" data-user-id="229913">@Andreas Molzer</span> you can try panicking if an allocation is done that is larger than EXP and then compile your program and backtrace the panic</p>



<a name="171314139"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/Extremely%20large%20statics/near/171314139" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> oli <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/Extremely.20large.20statics.html#171314139">(Jul 20 2019 at 06:16)</a>:</h4>
<p>Must be bigger than anything from libcore/libstd otherwise you can't get to your test</p>



<a name="171327916"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/Extremely%20large%20statics/near/171327916" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> HeroicKatora <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/Extremely.20large.20statics.html#171327916">(Jul 20 2019 at 13:40)</a>:</h4>
<p>I think the answer might just be 'somewhere in llvm'. I've tried to add a panic and it didn't trigger.</p>



<a name="171327930"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/Extremely%20large%20statics/near/171327930" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> HeroicKatora <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/Extremely.20large.20statics.html#171327930">(Jul 20 2019 at 13:41)</a>:</h4>
<p>Being able to compile with usage of a static <code>MaybeUninit&lt;[u8; 30]&gt;</code> (raw and in in a const function) is already a major win so I wouldn't worry too much.</p>



<a name="171328604"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/Extremely%20large%20statics/near/171328604" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> HeroicKatora <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/Extremely.20large.20statics.html#171328604">(Jul 20 2019 at 14:00)</a>:</h4>
<p>Running the slightly less ridiculous <code>[u8; 1 &lt;&lt; 34]</code> works fine until I hit: <code>LLVM ERROR: out of memory</code></p>



<a name="171585592"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/Extremely%20large%20statics/near/171585592" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> HeroicKatora <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/Extremely.20large.20statics.html#171585592">(Jul 24 2019 at 07:52)</a>:</h4>
<p><span class="user-mention" data-user-id="124288">@oli</span> Can I get another review? I'm done with the main work on mir and llvm, but there are apparently two target-specific workarounds that depend on the byte representation and for which I am not completely sure on how to adjust them.</p>



<a name="171597187"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/Extremely%20large%20statics/near/171597187" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> oli <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/Extremely.20large.20statics.html#171597187">(Jul 24 2019 at 11:18)</a>:</h4>
<p>we can always default to the previous behaviour irrelevant of the memory usage and correctness of the previous behaviour for these targets</p>



<a name="171599751"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/Extremely%20large%20statics/near/171599751" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> HeroicKatora <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/Extremely.20large.20statics.html#171599751">(Jul 24 2019 at 12:04)</a>:</h4>
<p>The wasm code requires a pointer, probably to a region of the allocation length. Since we can't simply allocate into the <code>&amp;Allocation</code> (requires  mutable) I'm just unsure if an entirely new allocation (<code>vec![0; length]</code>) would also suffice or if it requires a lifetime beyond the pointer use.</p>



<a name="171600085"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/Extremely%20large%20statics/near/171600085" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> oli <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/Extremely.20large.20statics.html#171600085">(Jul 24 2019 at 12:10)</a>:</h4>
<p>I don't think it requires a lifetime longer than the function call</p>



<a name="171600112"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/Extremely%20large%20statics/near/171600112" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> oli <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/Extremely.20large.20statics.html#171600112">(Jul 24 2019 at 12:10)</a>:</h4>
<p>this is one of the places where the duplication comes from</p>



<hr><p>Last updated: Aug 07 2021 at 22:04 UTC</p>
</html>